Through in-context learning (ICL), large-scale language models are effective few-shot learners without additional model fine-tuning. However, the ICL performance does not scale well with the number of available training samples as it is limited by the inherent input length constraint of the underlying language model. Meanwhile, many studies have revealed that language models are also powerful feature extractors, allowing them to be utilized in a black-box manner and enabling the linear probing paradigm, where lightweight discriminators are trained on top of the pre-extracted input representations. This paper proposes prompt-augmented linear probing (PALP), a hybrid of linear probing and ICL, which leverages the best of both worlds. PALP inherits the scalability of linear probing and the capability of enforcing language models to derive more meaningful representations via tailoring input into a more conceivable form. Throughout in-depth investigations on various datasets, we verified that PALP significantly enhances the input representations closing the gap between ICL in the data-hungry scenario and fine-tuning in the data-abundant scenario with little training overhead, potentially making PALP a strong alternative in a black-box scenario.
translated by 谷歌翻译
大规模的预训练的语言模型(PLM)以能够仅通过在提示中调节一些被称为示范的示威演示的情况而不明确调整为所需的下游任务而被称为示威的示威来解决任务。但是,这种过程(即,在文章中的学习)自然会高度依赖通常从外部数据集中选择的演示。在本文中,我们提出了自我生成的文化学习(SG-ICL),该学习生成了从PLM本身中的文化学习演示,以最大程度地减少对外部演示的依赖。我们对四个不同的文本分类任务进行实验,并显示SG-ICL的表现明显优于零射击学习,并且通常价值约0.6个黄金训练样本。此外,与培训数据集的随机选择相比,我们的生成的演示表现出更一致的性能,方差较低。
translated by 谷歌翻译
According to the rapid development of drone technologies, drones are widely used in many applications including military domains. In this paper, a novel situation-aware DRL- based autonomous nonlinear drone mobility control algorithm in cyber-physical loitering munition applications. On the battlefield, the design of DRL-based autonomous control algorithm is not straightforward because real-world data gathering is generally not available. Therefore, the approach in this paper is that cyber-physical virtual environment is constructed with Unity environment. Based on the virtual cyber-physical battlefield scenarios, a DRL-based automated nonlinear drone mobility control algorithm can be designed, evaluated, and visualized. Moreover, many obstacles exist which is harmful for linear trajectory control in real-world battlefield scenarios. Thus, our proposed autonomous nonlinear drone mobility control algorithm utilizes situation-aware components those are implemented with a Raycast function in Unity virtual scenarios. Based on the gathered situation-aware information, the drone can autonomously and nonlinearly adjust its trajectory during flight. Therefore, this approach is obviously beneficial for avoiding obstacles in obstacle-deployed battlefields. Our visualization-based performance evaluation shows that the proposed algorithm is superior from the other linear mobility control algorithms.
translated by 谷歌翻译
While witnessing the noisy intermediate-scale quantum (NISQ) era and beyond, quantum federated learning (QFL) has recently become an emerging field of study. In QFL, each quantum computer or device locally trains its quantum neural network (QNN) with trainable gates, and communicates only these gate parameters over classical channels, without costly quantum communications. Towards enabling QFL under various channel conditions, in this article we develop a depth-controllable architecture of entangled slimmable quantum neural networks (eSQNNs), and propose an entangled slimmable QFL (eSQFL) that communicates the superposition-coded parameters of eS-QNNs. Compared to the existing depth-fixed QNNs, training the depth-controllable eSQNN architecture is more challenging due to high entanglement entropy and inter-depth interference, which are mitigated by introducing entanglement controlled universal (CU) gates and an inplace fidelity distillation (IPFD) regularizer penalizing inter-depth quantum state differences, respectively. Furthermore, we optimize the superposition coding power allocation by deriving and minimizing the convergence bound of eSQFL. In an image classification task, extensive simulations corroborate the effectiveness of eSQFL in terms of prediction accuracy, fidelity, and entropy compared to Vanilla QFL as well as under different channel conditions and various data distributions.
translated by 谷歌翻译
Neural networks trained with ERM (empirical risk minimization) sometimes learn unintended decision rules, in particular when their training data is biased, i.e., when training labels are strongly correlated with undesirable features. To prevent a network from learning such features, recent methods augment training data such that examples displaying spurious correlations (i.e., bias-aligned examples) become a minority, whereas the other, bias-conflicting examples become prevalent. However, these approaches are sometimes difficult to train and scale to real-world data because they rely on generative models or disentangled representations. We propose an alternative based on mixup, a popular augmentation that creates convex combinations of training examples. Our method, coined SelecMix, applies mixup to contradicting pairs of examples, defined as showing either (i) the same label but dissimilar biased features, or (ii) different labels but similar biased features. Identifying such pairs requires comparing examples with respect to unknown biased features. For this, we utilize an auxiliary contrastive model with the popular heuristic that biased features are learned preferentially during training. Experiments on standard benchmarks demonstrate the effectiveness of the method, in particular when label noise complicates the identification of bias-conflicting examples.
translated by 谷歌翻译
在自主驾驶环境中,同时保证实时和准确的对象检测至关重要。但是,现有的对象检测神经网络系统的特征是计算时间和准确性之间的权衡,因此必须优化这种权衡。幸运的是,在许多自动驾驶环境中,图像以连续的形式出现,提供了使用光流的机会。在本文中,我们利用光流估计来提高对象检测神经网络的性能。此外,我们提出了一个lyapunov优化框架,以实现稳定性的时间平均性能最大化。它可以自适应地确定是否使用光流程适合动态车辆环境,从而确保车辆的队列稳定性和同时的时间平均最高性能。为了验证关键思想,我们使用各种对象检测神经网络和光流估计网络进行数值实验。此外,我们通过Yolov3微小和Flownet2-S展示了可自配置的稳定检测,它们分别是实时对象检测网络和光流估计网络。在演示中,我们提出的框架将准确性提高了3.02%,检测到的对象数量增加了59.6%,并且用于计算功能的队列稳定性。
translated by 谷歌翻译
随着嘈杂的中间量子量子(NISQ)时代的开始,量子神经网络(QNN)最近已成为解决经典神经网络无法解决的问题的解决方案。此外,QCNN吸引了作为下一代QNN的注意力,因为它可以处理高维矢量输入。但是,由于量子计算的性质,经典QCNN很难提取足够数量的功能。在此激励的情况下,我们提出了一种新版本的QCNN,称为可伸缩量子卷积神经网络(SQCNN)。此外,使用QC的保真度,我们提出了一种名为ReverseDelity Trainity(RF-Train)的SQCNN培训算法,可最大程度地提高SQCNN的性能。
translated by 谷歌翻译
网络安全研究中的关键主题之一是自动COA(行动)攻击搜索方法。被动搜索攻击的传统COA攻击方法可能很困难,尤其是随着网络变大。为了解决这些问题,正在开发新的自动COA技术,其中,本文设计了一种智能的空间算法,以在可扩展网络中有效运行。除空间搜索外,还考虑了基于蒙特卡洛(MC)的时间方法来照顾时间变化的网络行为。因此,我们为可扩展和时变网络的时空攻击COA搜索算法提出了一个时空攻击。
translated by 谷歌翻译
尽管量子至高无上尚未到来,但最近在实用量子计算的迫在眉睫的时代,人们对​​确定量子机学习的潜力(QML)的兴趣越来越大。由此激励,在本文中,我们基于具有两个单独的可训练参数的单独维度的量子神经网络(QNN)的独特特征重新设计多代理增强学习(MARL):影响输出Qubit状态和极点参数的角度参数:与输出测量基础相关。我们提出了将这种二元训练性作为元学习能力,我们提出了量子元marl(QM2ARL),该量子元MARL(QM2ARL)首先应用角度训练进行元学习,然后进行极点训练,以进行几次射击或局部QNN培训。为了避免过度拟合,我们在角度训练期间开发了一种将噪声注入到极域中的角度正则化技术。此外,通过将极点作为每个受过训练的QNN的内存地址利用,我们介绍了极点内存的概念,允许仅使用两参数极点值保存和加载经过训练的QNN。从理论上讲,我们证明了角度到极正则化下的角度训练的收敛性,并通过模拟证实了QM2ARL在获得高奖励和快速收敛方面的有效性,以及在快速适应时间变化环境中的极点记忆。
translated by 谷歌翻译
最近的深度学习模型在言语增强方面已经达到了高性能。但是,获得快速和低复杂模型而没有明显的性能降解仍然是一项挑战。以前的知识蒸馏研究对言语增强无法解决这个问题,因为它们的输出蒸馏方法在某些方面不符合语音增强任务。在这项研究中,我们提出了基于特征的蒸馏多视图注意转移(MV-AT),以在时域中获得有效的语音增强模型。基于多视图功能提取模型,MV-AT将教师网络的多视图知识传输到学生网络,而无需其他参数。实验结果表明,所提出的方法始终提高瓦伦蒂尼和深噪声抑制(DNS)数据集的各种规模的学生模型的性能。与基线模型相比,使用我们提出的方法(一种用于有效部署的轻巧模型)分别使用了15.4倍和4.71倍(FLOPS),与具有相似性能的基线模型相比,Many-S-8.1GF分别达到了15.4倍和4.71倍。
translated by 谷歌翻译